Skip to content

fix(controller): do not apply driver upgrade annotation when driver is disabled#1981

Merged
tariq1890 merged 2 commits intoNVIDIA:mainfrom
friedrice99:fix/driver-upgrade-annotation-when-driver-disabled
Feb 20, 2026
Merged

fix(controller): do not apply driver upgrade annotation when driver is disabled#1981
tariq1890 merged 2 commits intoNVIDIA:mainfrom
friedrice99:fix/driver-upgrade-annotation-when-driver-disabled

Conversation

@friedrice99
Copy link
Copy Markdown

The applyDriverAutoUpgradeAnnotation() function was applying the nvidia.com/gpu-driver-upgrade-enabled annotation to GPU nodes even when driver.enabled=false.
This occurred because the function only checked if driver.upgradePolicy.autoUpgrade was true, without verifying that the driver component itself was enabled.

This fix adds a check for Driver.IsEnabled() before applying the annotation, ensuring it is only set when:

  1. Driver is enabled
  2. Auto-upgrade policy exists and is enabled
  3. Sandbox workloads are disabled

Added unit tests to validate the fix and prevent regression.

Fixes #1277

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Dec 10, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@rajathagasthya
Copy link
Copy Markdown
Contributor

/ok-to-test b0249cf

Comment thread controllers/state_manager_test.go Outdated
@rajathagasthya rajathagasthya force-pushed the fix/driver-upgrade-annotation-when-driver-disabled branch 2 times, most recently from 1f9cf4b to 40a6f7b Compare February 13, 2026 15:10
@rajathagasthya
Copy link
Copy Markdown
Contributor

/ok-to-test 40a6f7b

Anuj Dube and others added 2 commits February 19, 2026 17:44
…s disabled

The applyDriverAutoUpgradeAnnotation() function was applying the
nvidia.com/gpu-driver-upgrade-enabled annotation to GPU nodes even
when driver.enabled=false. This occurred because the function only
checked if driver.upgradePolicy.autoUpgrade was true, without
verifying that the driver component itself was enabled.

This fix adds a check for Driver.IsEnabled() before applying the
annotation, ensuring it is only set when:
1. Driver is enabled
2. Auto-upgrade policy exists and is enabled
3. Sandbox workloads are disabled

Added unit tests to validate the fix and prevent regression.

Fixes NVIDIA#1277

Signed-off-by: Anuj Dube <anujdubesn@gmail.com>
Signed-off-by: Rajath Agasthya <ragasthya@nvidia.com>
@rajathagasthya rajathagasthya force-pushed the fix/driver-upgrade-annotation-when-driver-disabled branch from 40a6f7b to bf5763f Compare February 19, 2026 23:46
@rajathagasthya
Copy link
Copy Markdown
Contributor

/ok-to-test bf5763f

@tariq1890 tariq1890 merged commit 691bf06 into NVIDIA:main Feb 20, 2026
16 checks passed
@friedrice99 friedrice99 deleted the fix/driver-upgrade-annotation-when-driver-disabled branch March 9, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nvidia.com/gpu-driver-upgrade-enabled: "true" even when the driver.enabled=false

3 participants